EXHIBIT A 



Please delete the specification in its entirety and replace with the following: 



--METHOD AND SYSTEM FOR THREE-DIMENSIONAL HANDWRITING 
5 RECOGNITION 

BACKGROUND OF THE INVENTION 

10 The invention relates generally to handwriting recognition technologies, and 

more particularly to three-dimensional (3D) handwriting recognition methods and 
systems. 

Handwriting recognition is a technique by which an intelligence system can 
15 recognize characters and other symbols written by hand. This technique has been 
very popular since it frees users from the keyboard, allowing users to write and draw in 
a more natural way. With the increasing demands from users, more and more devices 
now have incorporated handwriting recognition system to give users natural input 
experience. Handwriting recognition is particularly popular in various handheld 
20 devices that provides handwriting recognition of words with complex structures, e.g., 
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Chinese characters, input of which using a conventional keyboard is very time 
consuming. 

A handwriting input device provides a user with a friendly way of inputting 
5 information. At present, the minimum requirement for an inputting device is a mouse. 
To write with the mouse, the user needs to press and hold down the primary mouse 
button, and then move the mouse pointer to form strokes of a word or character for 
generating the final word or character. Popular handwriting input devices, such as a 
pen stylus and tablet, are used on conventional handheld devices such as PDAs, or 
10 are connected to a computer through a serial or USB port. Handheld devices often 
use a pen stylus and a touch screen as a pad to allow users to perform recognition 
functions. Most handheld devices, such as PDAs, are equipped with this kind of input 
device. 

15 Another kind of handwriting input device includes a pen that allows users to 

transfer data into a receiving unit, such as a cellular phone, a PDA or a PC by simply 
writing or drawing in their own natural handwriting on the comfort and space of a 
regular piece of paper. 

20 At present, all conventional handwriting input devices adopt a two-dimensional 

input method. Users have to write on a physical medium, such as a tablet, a touch 
screen, or a paper pad. This limits the choices available for users. For example, if a 
user wants to write some comments during a presentation or a lecture, he or she 
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would have to first find a physical medium, e.g., a table, a paper pad, etc. This can 
cause much inconvenience for the user while standing in the room, giving the 
presentation or lecture. Also, in a mobile environment, such as in a car, a bus or a 
subway, it would be very inconvenient to "write" on a physical medium using a stylus. 

Therefore, there is a need to provide an improved handwriting recognition 
system that gives users more flexibility and convenience and frees the users from the 
physical medium required for two-dimensional handwriting recognition. 
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SUMMARY OF THE INVENTION 



The present invention gives users more flexibility and enjoyable writing 
experience by allowing users to freely write words or characters in a 3D space in a 
touchless way, without requiring any physical medium such as a pad or a tablet. 

5 

According to the invention, there is provided a handwriting recognition system 
that comprises an input device and a recognition device in communication with the 
input device. The input device includes a three-dimensional (3D) motion detection 
sensor configured to generate 3D motion data in response to a 3D motion. In one 
10 embodiment, the motion detection sensor measures acceleration of the 3D motion in 
X, Y and Z axial directions to generate the 3D motion data. The recognition device is 
configured to receive (e.g., by wireless means) the 3D motion data from the input 
device and derive corresponding two-dimensional (2D) images for handwriting 
recognition, based on the 3D motion data. 

15 According to one embodiment of the invention, the recognition device calculates 

corresponding 3D coordinates based on the 3D motion data, constructs corresponding 
3D tracks based the 3D coordinates, and derives the corresponding 2D images from 
the 3D tracks by mapping the 3D tracks onto a 2D plane for handwriting recognition. 

Other objects and attainments together with a fuller understanding of the 
20 invention will become apparent and appreciated by referring to the following 
description and claims taken in conjunction with the accompanying drawings. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



The invention is explained in further detail, and by way of example, with 
reference to the accompanying drawings wherein: 

FIG. 1 shows a three-dimensional handwriting recognition system according to 
5 one embodiment of the invention; 

FIG. 2 is a flow chart diagram illustrating a recognition process according to one 
embodiment of the invention; 

FIG. 3A shows a 2D image of a Chinese character derived by mapping 3D 
tracks onto a 2D projection plane; 

10 FIG. 3B shows a final result of handwriting recognition process based on the 2D 

image in FIG. 3A; 

FIG. 4 shows an external design of a 3D handwriting input device according to 
one embodiment of the invention; and 

FIG. 5 illustrates how the input device may be mounted. 

15 Throughout the drawings, the same reference numerals indicate similar or 

corresponding features or functions. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 



FIG. 1 shows a three-dimensional handwriting recognition system 10 according 
to one embodiment of the invention. As illustrated, system 10 comprises a handwriting 
input device 20, a recognition device 30 and an output device 40. Input device 20 
5 includes a 3D motion detection sensor 22, a control circuit 26 and a communication 
interface 28. Recognition device 30 includes a processor 32, a memory 34, a storage 
device 36, and a communication interface 38. For simplicity, other conventional 
elements are not shown in FIG. 1 . 

10 In operation, a user moves input device 20 to freely write words or characters in 

a 3D space, e.g., in the air. Motion detection sensor 22 detects the 3D motion and 
communicates the 3D motion data and a sampling rate to recognition device 30 for 
handwriting recognition via a communication interface 28, such as Bluetooth, Zigbee, 
IEEE 802.11, infrared, or a USB port. The sampling rate may be a predetermined 

15 value set by an end user or a manufacturer based on factors such as the processing 
capability of the system. Alternatively, the sampling rate may be dynamically 
determined and adjusted based on, for example, the speed of the motion. This can be 
done, for example, by first determining the speed of the initial motion associated with 
the handwriting. Then the recognition device can dynamically adjust the sampling rate 

20 based on the speed of the motion at the last sampling point. The higher the speed is, 
the higher the sampling rate will be, and vice versa. By dynamically adjusting the 
sampling rate, the accuracy of the handwriting recognition can be improved since only 



the optimal number of sampling points will be used for constructing the word or 
character. Further, lower power consumption is needed. 

Based on the motion data and the sampling rate received from input device 20, 
5 processor 32 calculates the corresponding 3D coordinates on the X, Y and Z axes and 
stores these coordinates in storage device 36. Using the calculated coordinates, 
processor 32 constructs the corresponding 3D tracks. The 3D tracks will then be 
projected onto a 2D plane to form 2D images which will be recognized using 
conventional handwriting recognition software. The final result is displayed on output 
10 device 40. 

Since 3D writing is a continuous process, control circuit 26 of input device 20 
provides a control signal to recognition device 30 via interface 28 to indicate 
separation of individual words or characters upon receiving a user provided external 
15 input. For example, the user may press a control button to cause control circuit 26 to 
generate the control signal after completion of writing a word or a character. 

According to a specific embodiment of the invention, motion detection sensor 
22 detects the 3D motion by measuring the acceleration of the movement along the X, 
20 Y and Z axes. As an example, the piezoresistive-type tri-axial accelerating sensor 
commercially available from Hitachi Metals, Ltd., Tokyo, Japan, may be used as 
motion detection sensor 22. This accelerating sensor in the form of an IC chip has the 
ability to simultaneously detect acceleration in the three axial directions (X, Y and Z). 
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The senor is highly sensitive and shock resistant and is a very small and thin 
semiconductor type 3 axial accelerating sensor. More information about this 
accelerating sensor is available on the following website http://www.hit.achi- 
metals.co.jp/e/prod/prod06/p06_10.html, which is hereby incorporated by reference. 

5 

FIG. 2 is a flow chart diagram illustrating a recognition process 100 performed 
by recognition device 30, according to one embodiment of the invention. In FIG. 2, 
recognition device 30 receives the 3D motion data (e.g., the acceleration data of the 
movement in the X, Y and Z directions) and the sampling rate from input device 20 
10 (step 102). Based on the information received, processor 32 calculates the 
corresponding 3D coordinates on the X, Y and Z axes for each sampling point using 
the starting point of the movement as the origin (step 106). Each sampling point is 
also used as a reference point for calculating the coordinates of the following sampling 
point. 

15 

Calculation of the 3D coordinates is continuously performed based on the 
incoming 3D motion data until processor 32 detects receipt of a control signal (step 
112). The control signal indicates completion of writing a word or a character. The 
corresponding 3D tracks are constructed using the 3D coordinates (step 116) and are 
20 then mapped onto a 2D plane (step 122). Thereafter, conventional 2D handwriting 
recognition is performed (step 126). 



In the above recognition process 100, to map the 3D tracks onto a 2D plane at 
step 122, it is necessary to first find a proper 2D projection plane. In a preferred 
embodiment of the invention, a proper 2D projection plane is separately derived for 
each word or character. 

5 

According to the geometry principles, a proper 2D projection plane is a plane, to 
which the sum of the distance square of each sampling point is minimal. Assume the 
coordinates of n sampling points are known as follows: 

(x l ,y l ,z l ),(x 2 ,y 2 ,z 2 )...(x„,y„,z tt ) , and the equation of the plane is 
10 Ax+By+Cz+D = 0, where (A 2 +B 2 +C 2 *0) . Now it is necessary to determine the 
values of A,B,C,D. The distance from one point (ipVpZ,) to the plane is given as 
d ^ = \Ax l +By } +Cz l +D\ ^ ^ ^ ^ £ j2 represented by F ^ BiC ,D) is 

j a 2 +b 2 + c 2 m 
expressed as: 

MARrm W +B% +Cz +Df +{Ax l +By_ +Cz_ +D) 2 +...+(Ax, +By n +Cz h +D) 2 
F(A,B,C,D)=2_d i = -J^C 
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The values of A,B,C,D can be determined using the Lagrange multiplication 
method as follows, which is described in Mathematics Analysis by Ouyang 
Guangzhong, published by Fudan University Press in 1999 in China, which is hereby 
incorporated by reference. Thus, 

20 F(A, B,C,D) = F' (A, B, C, D) = (Ax, + By, + Cz x + Df + (Ax 2 + By 2 + Cz 2 + Df + ... + (Ax n + By n + Cz n + Df 



under the constraint A 2 +B 2 +C 2 = 1. From this equation, the following equation is 
derived: 

G(A, B, C, D) = F'(A, B, C, D) + A(A 2 +B 2 +C 2 -\), 
where X is a Lagrange multiplier, which is a constant. The partial differential functions 
are now performed on G(A,B,C,D) with respect to A, B, C and D as follows: 

dG(A,B,C,D) _ Q 

8A 

8G(A,B,C,D) _ Q 
8B 

8G(A,B,C,D) _ Q 

ac 

From the above four equations, the following equations are obtained: 



A Hl to *y,)+ B (Lly (y t **)+*)+ ^EL 0* * Z J + D Hl x = 0 ( 2 ) 

A T i= S x i **i)+*E>i *yd+c(Y i= S z i* z i) + v +D Y i= , z i=<> ( 3 ) 

^E>* + *E><- +c Z-=i^ =0 ( 4 ) 

A 2 +B 2 +C 2 =\ (5) 
where equation (4) can be rewritten as: 



By incorporating equation (6) into equations (1), (2), and (3), the following is 
obtained: 
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^fo^—EA**) EA^—EA**,) EA^—EA**') 



J" 




"J" 






=-1 




(7) 


c 




c 





Thus, from the above equations, the values of A, B, C and D can be obtained. 



As an example, the following 3D coordinates for a total of 13 sampling points 
5 are obtained for a Chinese character: 

{(0,0,0),(0.49,-1,0.02),(1,0,0.03),(0,0,0.02),(2,-0.13,0.01),(3.5,-0.14,0),(3.76,-0.31,- 
0.01 ),(2.74,-1 .26,0.01 ),(1 .38,-2.24,0),(2.5,-2,0.01 ),(1 .746,1 ,0.02,),(2,-1 .5,0.03),(1 .876,- 
4.5,0.02),}. 



10 By using the Lagrange multiplication method described above, the following are 
obtained: 



53.407 


-23.6725 


0.1911 " 








~A 


23.6725 


36.2195 


-0.2084 




B 


= -A 


B 


0.1911 


- 0.2084 


0.0035 




C 




C 



A 2 +B 2 +C 2 =l,and 

D = -^(A^x i+ B^y i+ C^ Zi ). 

15 From the above equations, the values of A, B, C and D are determined as follows: 

A = 0.0045, B = 0.0023, C = 0.9999, D = -0.01777 . 



Thus, the equation of the 2D projection plane is 
0.0045x + 0.0023j + 0.9999z-0.01777 = 0. From the equation of the projection plane, 
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Ax + By + Cz + D = 0, and the equation of a line that is vertical to the projection plane, 

x * l = - -jp- = z ^ , the following equations are derived: 

_ (B 2 + C 2 )x. - A{By t + Cz i + D) 
X ~ A 2 +B 2 +C 2 

_ (A 2 + C 2 )y i - B(Ax i + Cz t + D) 
y ~ A 2 +B 2 +C 2 

5 which can be used to obtain the corresponding 2D coordinates for each 3D sampling 

point. In this example, the following corresponding 2D coordinates are obtained: 

{(0.0001 ,0),(0.49,-1 ),(1 ,-2.0001 ),(0,0),(2,-0.1 3),(3.5001 ,-0.1 4),(3.7601 ,-0.31 ),(2.7401 ,- 

1 .26),(1 .3801 ,2.24),(2.5001 ,-2),(1 .746,1 ),(2,-1 .5001 ),(1 .876,-4.5001 )}. 

Based on these 2D coordinates, the corresponding 2D image is projected onto the 2D 

10 projection plane. The result of this Chinese character is shown in FIG. 3A. The final 

result is generated by performing a conventional 2D handwriting recognition process, 

as shown in FIG. 3B. 

In the above, the Lagrange multiplication method is used to obtain the values of 
15 A, B, C and D. Other methods such as linear regression method may also be used. 

FIG. 4 shows an external design of a 3D handwriting input device 200 
according to one embodiment of the invention. As shown in FIG. 4, input device 200 
includes a housing 210 that contains the electronics parts of the device (such as a 3D 
20 motion detection sensor IC chip), a control button 212 for allowing a user to input a 
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control signal to indicate completion of writing a word or character, and a band 220 for 
mounting input device 200 on the user's finger. 

FIG. 5 illustrates how input device 200 may be mounted. In FIG. 5, device 200 
5 is mounted on a finger 232 of a user's hand 230. By mounting it on the finger, the user 
can simply move the finger to write any word or character in a 3D space. The 3D 
motion data will be wirelessly transmitted to a recognition device for handwriting 
recognition. 

10 According to the invention, the input device and the recognition device can be 

integrated together as a single unit (e.g., a pen) that operates in the same manner as 
described above. The final recognition results will be transmitted to an intelligent 
apparatus such as a PDA, a laptop computer, a PC, etc. 

15 The invention can also be used on a 2D plane if the user so chooses. In such a 

case, the coordinates calculated will be for 2D plane and the step of mapping the 3D 
tracks onto a 2D plane is omitted. 

While the invention has been described in conjunction with specific 
20 embodiments, it is evident that many alternatives, modifications and variations will be 
apparent to those skilled in the art in light of the foregoing description. Accordingly, it 
is intended to embrace all such alternatives, modifications and variations as fall within 
the spirit and scope of the appended claims. 
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